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Abstract 


Recognizing  the  limitations  of  its  decades-old  legacy  information  technology  systems,  the 
U.S.  Internal  Revenue  Service  began  an  effort  to  modernize.  In  1998,  after  two  initial  efforts 
to  modernize  achieved  limited  results,  the  IRS  established  a  partnership  with  the  private  sec¬ 
tor  and  awarded  a  contract  to  the  PRIME  Alliance  which  was  designed  to  assume  the  devel¬ 
opment  and  integration  role.  At  that  time,  the  IRS  Business  Systems  Modernization  Office 
(BSMO)  was  established  to  manage  the  program. 


However,  as  with  past  modernization  efforts,  the  BSMO  had  difficulties  in  developing  the 
discipline  to  efficiently  and  effectively  manage  the  acquisition  aspects  of  this  modernization 
effort.  The  General  Accounting  Office  suggested  that  the  BSM  program  instill  this  discipline 
by  improving  a  number  of  management  process  areas,  including  its  acquisition  processes. 
This  suggestion  included  application  of  the  Software  Acquisition  Capability  Maturity  Model 
(SA-CMM)  as  guidance  on  how  to  improve. 


This  paper  provides  an  overview  of  applying  the  SA-CMM  to  the  IRS  modernization  effort  to 
establish  and  implement  more  effective  acquisition  management  processes  and  practices.  The 
experience  includes  the  process  improvement  planning  stages  of  first  selecting  the  SA-CMM 
as  a  framework  for  process  improvement,  through  to  completion  of  the  final  assessment 
where  a  maturity  level  2  rating  was  achieved  against  the  SA-CMM. 
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1  Introduction 


Increasingly,  many  federal  organizations  are  reevaluating  the  way  they  do  business.  There  are 
numerous  reasons  for  this  revamping,  but  the  driving  force  is  the  need  is  to  comply  with  con¬ 
gressional  direction.  Such  revamping  has  been  referred  to  as  business  modernization  or  en¬ 
terprise  modernization.  Based  on  such  congressional  direction,  the  U.S.  Internal  Revenue 
Service  (IRS)  has  undertaken,  over  the  last  decade,  business  modernization  of  its  legacy  sys¬ 
tems  and  the  way  it  does  business  for  the  U.S.  taxpayer. 


The  U.S.  federal  tax  system  collects  more  than  $2  trillion  in  gross  revenue  each  year,  or  over 
95%  of  the  government’s  revenue.  The  tax  system  is  dependent  on  a  collection  of  computer 
systems  and  resident  software  that  evolved  over  35  years.  Nearly  all  IRS  employees  depend 
on  these  computer  systems  to  do  their  daily  jobs,  including  more  than  70,000  who  use  these 
systems  to  deliver  direct  service  to  taxpayers.  Maintaining  these  systems  year  after  year  takes 
skilled  and  dedicated  people.  But  there  are  problems  they  could  and  cannot  fix. 


Recognizing  these  problems,  the  IRS  embarked  on  a  modernization  effort  over  a  decade  ago, 
with  limited  success.  Difficulties  in  this  effort  have  been  attributed  to  poor  management  of 
the  acquisitions  involved  [Glass  98],  The  General  Accounting  Office  (GAO)  suggested  that 
the  IRS  modernization  effort  be  contracted  to  an  integration  contractor  responsible  for  con¬ 
trolling  the  acquisition  and  development  efforts  on  behalf  of  the  IRS  to  put  a  new  tax  system 
in  place  [GAO  96].  In  this  referenced  report,  the  GAO  stated: 

“It  is  clear  that  unless  IRS  has  mature,  disciplined  processes  for  acquiring  soft¬ 
ware  systems  through  contractors,  it  will  be  no  more  successful  in  buying  soft¬ 
ware  than  it  has  in  building  software.  ” 


In  1998,  the  IRS  Business  Systems  Modernization  (BSM)  program  awarded  a  contract  to  the 
PRIME  Alliance,  led  by  Computer  Sciences  Corporation,  to  assume  this  development  and 
integration  role.  At  the  same  time,  the  IRS  Business  Systems  Modernization  Office  (BSMO) 
was  established  to  manage  the  program. 


Subsequently,  the  IRS  and  GAO  recognized  that  the  BSMO  itself  needed  the  discipline  to 
efficiently  and  effectively  manage  the  acquisition  aspects  of  managing  the  PRIME  Alliance 
contract.  The  GAO  suggested  that  the  BSM  program  instill  this  discipline  and  improve  a 
number  of  management  process  areas,  including  its  acquisition  processes.  This  suggestion 
included  the  application  of  the  Software  Acquisition  Capability  Maturity  Model  (SA-CMM) 
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stewarded  by  the  Carnegie  Mellon®  Software  Engineering  Institute  (SEI)  as  guidance  on  how 
to  improve.  BSMO  management  understood  the  importance  of  having  sound  acquisition 
management  practices  in  place,  and  the  BSMO  embarked  on  a  process  improvement  mission 
using  the  SA-CMM  as  the  foundation  model.  (  Appendix  A  contains  a  brief  overview  of  the 
SA-CMM.) 


This  paper  provides  an  overview  of  the  approach  and  experiences  of  applying  the  SA-CMM 
to  the  IRS  modernization  effort  to  establish  more  effective  acquisition  management  processes 
and  practices.  Chapter  2  discusses  the  context  behind  the  BSMO  process  improvement  effort 
and  the  motivation  to  use  the  SA-CMM.  Chapter  3  provides  a  brief  description  of  the  process 
improvement  approach  taken,  relying  on  the  typical  principles  of  process  improvement. 
Chapter  4  gives  the  reader  an  overview  of  the  results.  In  Chapter  5  we  summarize  the  process 
improvement  effort  and  provide  lessons  learned. 


Carnegie  Mellon  is  registered  in  the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon  Uni¬ 
versity. 
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2  Background 


Modernization  projects  approved  by  IRS  executive  leadership  use  the  PRIME  Alliance  con¬ 
tract  to  acquire  the  products  and  services  constituting  the  modernization  program.  The  BSMO 
develops  and  issues  task  orders  under  this  contract  to  engage  the  Alliance  to  develop  and  in¬ 
tegrate  the  modernization  solutions. 


The  BSMO’s  own  realization  that  it  needed  to  improve  its  internal  discipline  to  efficiently 
and  effectively  manage  the  PRIME  Alliance  contract  was  reinforced  by  findings  of  oversight 
organizations  such  as  the  General  Accounting  Office.  In  a  GAO  report  [GAO  00]  to  the  U.S. 
Congress,  the  IRS  was  reported  to  be 

“an  agency  fraught  with  long-standing  and  significant  management  problems 
and  a  history  of  ineffective  attempts  to  correct  them.  ” 

“However,  IRS  must  overcome  several  serious  management  challenges  in  its  cur¬ 
rent  systems  modernization  effort  before  it  will  be  ready  to  build  modernized  sys¬ 
tems.  In  particular,  IRS  must  (1)  complete,  enforce,  and  maintain  an  enterprise 
systems  architecture,  (2)  establish  and  implement  sound  investment  management 
processes  to  ensure  only  incremental,  cost-effective  system  investments  are  made, 
and  (3)  impose  software  acquisition  and  life  cycle  management  discipline  on 
each  system  investment  it  undertakes.  ” 

More  emphasis  was  placed  on  BSMO  instilling  its  acquisition  management  and  process  dis¬ 
cipline  in  December  of  2001  when  the  GAO  recommended  delaying  or  reducing  IRS  mod¬ 
ernization  funding  unless  something  was  done  quickly  to  assess  and  improve  current  multi¬ 
million  dollar  modernization  projects.  The  GAO  also  recognized  that  federal  acquisition 
agencies,  such  as  the  BSMO,  needed  an  approach  to  instill  discipline  and  take  responsibility 
for  managing  their  acquisitions.  The  following  exceipt  from  a  GAO  briefing  reflects  the  find¬ 
ing: 


“ Some  federal  agencies  have  been  using  innovative  ‘partnership  ’  arrangements 
with  the  contractor  for  IT  development,  assuming  that  the  contractor  is  responsi¬ 
ble  for  any  failures  or  missed  schedules.  This  is  usually  caused  by  the  federal 
agency  using  contractors  in  the  role  of  employees,  and  even  putting  contractors 
in  a  position  of  approving  their  own  products.  The  agencies  assume  that  a  part¬ 
nership  arrangement  absolves  them  of  all  acquisition  management  responsibili- 
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ties.  These  agencies  have  problems  above  those  that  can  be  pinpointed  by  the 
CMMs.  However,  a  strict  application  of  SA-CMM  can  provide  a  way  out  of  the 
confusion  caused  by  poorly  defined  roles  and  responsibilities  that  are  typical  in 
agency-contractor  partnership  arrangements.  Once  the  agency  identifies  the 
‘ acquiring  organization  ’and  that  acquiring  organization  is  deemed  responsible 
for  the  SA-CMM KPAs,  the  situation  can  be  brought  under  control.  ”I 


In  keeping  with  the  GAO’s  findings,  the  BSMO  decided  to  improve  its  acquisition  processes 
using  the  SA-CMM  as  the  process  improvement  model  of  choice.  Executive  management 
attention  on  this  process  improvement  effort  intensified. 


This  was  an  important  step  for  the  already  established  BSMO  Process  Improvement  Team. 
There  was  now  a  compelling  business  reason  to  go  forward  that  everyone  clearly  understood, 
namely,  “If  we  don’t  do  SA-CMM-based  process  improvement  soon,  our  funds  are  in  jeop¬ 
ardy!” 


Here  are  some  other  reasons  the  BSMO  decided  to  follow  the  GAO  suggestion  of  applying 

the  SA-CMM.  The  SA-CMM 

•  is  based  on  “best”  processes  and  practices  and  is  recommended  by  oversight  entities  (e.g., 
the  GAO)  to  enhance  acquisition  management  accountability.  (The  GAO  is  responsible 
for  auditing  and  ensuring  that  the  IRS  has  implemented  and  is  following  the  SA-CMM 
for  process  improvement.) 

•  sets  a  framework  for  incremental  process  improvement 

•  enables  efficient  and  effective  project  life  cycle  management  of  solution  acquisitions, 
from  pre-award  to  transition  to  operations  and  maintenance 

•  supports  a  congressional  mandate  to  set  up  a  “world  class”  acquisition  organization  by 
following  SEIsm  CMM®  methodologies  [GAO  00] 


1  Madhav  Panwar,  Naba  Barkakati,  &  Suzanne  Bums,  “Using  the  SA-CMM  to  Improve  Acquisition 
Processes,”  SEI  Symposium  2000. 

SM  SEI  is  a  service  mark  of  Carnegie  Mellon  University. 

®  CMM  is  registered  in  the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon  University. 
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3  Process  Improvement  Approach 


Early  in  the  process  improvement  effort,  the  overall  business  objective  of  the  process  im¬ 
provement  project  was  articulated  by  the  chief  information  officer  to  make  the  BSMO  more 
“accountable,  predictable,  and  timely”  in  the  performance  of  its  mission.  This  chapter  sets  out 
some  general  principles  for  process  improvement.  Many  of  the  principles  can  be  found  in  a 
2002  article  by  Fisher,  Goethert,  and  Jones  [Fisher  02].  This  discussion  on  principles  is  fol¬ 
lowed  by  brief  descriptions  of  how  the  BSMO  applied  these  and  other  principles  through  the 
process  improvement  project. 


3.1  Process  Improvement  Principles 

These  principles  were  used  to  guide  the  BSMO  process  improvement  effort.  These  principles 
are  inextricably  linked.  It  requires  a  concerted  effort  on  each  to  ensure  that  the  principles  are 
applied  correctly,  especially  in  light  of  the  organization’s  business  practices. 


3.1.1  Stabilize  the  Environment  Before  Attempting  Process 
Improvement 

Process  improvement,  whether  using  the  SA-CMM  or  not,  is  best  done  within  the  context  of 
a  stable  environment.  Some  organizations  do  not  understand  their  acquisition  mission,  do  not 
have  an  organizational  structure  or  skills  to  support  this  mission,  or  are  evolving  their  acqui¬ 
sition  processes  as  they  learn  how  to  accomplish  mission  requirements.  For  example,  organi¬ 
zations  that  treat  acquisition  as  simply  managing  the  contractor  or  just  being  a  coordinator 
between  affected  groups  significantly  underestimate  the  challenges  of  program  acquisition. 


In  addition,  employment  of  the  SA-CMM  could  help  stabilize  the  management  and  leader¬ 
ship  through  the  institutionalization  features.  Of  course,  there  is  no  guarantee  this  will  occur 
if  the  management  is  constantly  changing  and  new  managers  have  to  be  indoctrinated  or 
trained  to  rethink  their  previous  experience  in  terms  of  process  improvement  and  the  use  of 
processes  that  are  in  effect. 


Management  at  all  levels  must  understand,  buy  into,  articulate,  and  reinforce  the  organiza¬ 
tion’s  vision  and  strategic  goals  if  it  hopes  to  bring  some  level  of  stability  that  allows  not  only 
process  improvement  but,  more  critically,  accomplishment  of  the  mission.  Additionally,  the 
organization’s  management  must  be  deeply  involved  in  resolving  issues  with  interactions  and 
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interfaces  with  groups  external  to  the  organization  (e.g.,  political  and  turf  battles)  to  ensure 
this  stabilization. 


3.1.2  Leadership 

Successful  process  improvement  depends  on  “good”  leadership.  Good  leadership  provides 
elements  such  as  vision,  sponsorship,  planning,  managing  change  of  culture,  commitment 
throughout  the  organization,  and  ensuring  awareness  of  process  improvement  throughout  the 
organization. 


3.1.3  Employ  Demonstrated  Expertise  in  Process 
Improvement 

In  many  cases,  the  acquisition  organization  has  little  knowledge  or  experience  with  the  SA- 
CMM  and  process  improvement  in  general.  This  means  the  organization  may  have  to  contract 
externally  for  acquisition  process  improvement  services.  If  that  is  the  acquisition  organiza¬ 
tion’s  approach,  it  needs  to  obtain  expertise  in  SA-CMM-based  process  improvement  that  can 
be  verified  and  demonstrated.  (Saying  one  is  an  expert  is  not  proving  one  is  an  expert.)  This 
expertise  is  especially  critical  in  the  interpretation  of  the  SA-CMM  in  the  context  of  the  ac¬ 
quisition  organization’s  environment  and  business  paradigm. 


3.1.4  Avoid  “Big  Bangs” — Start  Small 

Process  improvement  takes  time.  Attempting  to  execute  a  process  improvement  project  (es¬ 
pecially  if  starting  from  scratch)  that  is  all  encompassing  and  will  be  completed  in  six  months 
is  typically  unsuccessful  and  results  in  eliminating  the  effort.  Do  not  try  to  improve  every¬ 
thing  at  once;  obtain  long-term  sponsorship  and  commitments;  devote  sufficient  resources  to 
the  effort;  remember  to  plan  deployment  of  process  improvement  efforts  to  the  users;  use 
process  improvement  planning  as  an  example  to  other  teams  as  the  right  way  to  run  a  project. 


3.1.5  Use  the  SA-CMM  as  a  Starting  Point  (Applying  the 
Reference  Model  to  the  Organization) 

Acquisition  organizations  that  buy  software-intensive  systems  tend  to  believe  the  SA-CMM 
is  not  applicable  to  their  acquisition  processes,  since  they  buy  systems,  not  software.  One 
reason  the  SA-CMM  was  developed  was  to  ensure  that  acquiring  organizations  realize  the 
criticality  of  software  in  their  acquisition.  However,  in  general,  the  SA-CMM  can  be  success¬ 
fully  applied  to  most  acquisition  organizations  and  their  unique  processes  in  buying  solutions 
and  systems,  whether  these  involve  software  or  not. 
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Terminology 


Terminology  is  critical  to  the  success  of  not  only  applying  any  process  improvement  refer¬ 
ence  model  like  the  SA-CMM,  but  also  more  significantly  operating  consistently  across  the 
organization.  Terminology  contains  the  concepts  and  sets  the  context  for  how  the  organiza¬ 
tion  wants  to  do  business  and,  it  follows,  how  the  organization  may  apply  process  improve¬ 
ment.  Terminology  provides  the  bridge  among  the  organization’s  way  of  doing  business,  the 
process  improvement  work,  appraisals,  etc.  In  this  light,  having  a  glossary  of  terms  is  consid¬ 
ered  critical  to  the  success  of  the  organization.  But  having  a  glossary  and  ensuring  that  per¬ 
sonnel  know  and  try  to  use  the  terms  correctly  and  consistently  is  always  a  challenge. 

Interpret  the  Model  in  Light  of  Business  or  Mission  Needs 

The  SA-CMM  should  be  interpreted  in  the  context  of  the  business  or  mission  needs  of  the 
organization.  Effective  and  efficient  acquisition  processes  are  critical  to  successful  acquisi¬ 
tions,  but  the  quality  of  their  output  can  be  determined  only  in  the  context  of  the  business 
needs  of  the  organization.  The  SA-CMM  should  be  and  is  typically  tailored  or  adapted  to  fit 
the  organization;  the  organization  should  not  be  restructured  to  reflect  the  SA-CMM.  Organi¬ 
zations  have  to  interpret  the  SA-CMM  for  themselves,  not  adopt  what  others  have  done  with¬ 
out  modification  of  how  the  organization  performs  its  mission;  i.e.,  the  way  it  does  business. 
Applying  the  SA-CMM  to  help  process  improvement  requires  professional  judgment,  intelli¬ 
gence,  and  common  sense.  Critically,  organizations  that  successfully  employed  the  SA-CMM 
in  their  process  improvement  efforts  learned  that  use  of  the  model  should  help  how  they  do  or 
want  to  do  business,  not  as  an  all-encompassing  mandate. 

Prematurely  Mandating  Achievement  of  a  Specific  CMM  Level 
Leads  to  Failure 

Some  acquisition  managers  have  little  knowledge  of  CMMs  and  their  use  in  process  im¬ 
provement.  Many  have  no  problem  imposing  CMM  levels  from  the  Capability  Maturity 
Model  for  Software  (SW-CMM)  or  other  models  upon  their  contractors,  but  do  not  know 
what  imposition  of  CMM  levels  on  their  internal  efforts  would  entail,  i.e.,  how  long  it  takes, 
what  it  takes,  needed  cultural  changes,  and  resources  required.  Instead,  managers  may  “draw 
a  line  in  the  sand”  and  announce  that  their  organization  will  achieve  SA-CMM  level  2  or  3  in 
six  months.  This  arbitrary  imposition  of  a  CMM  timeline  encourages  organizational  shortcuts 
in  process  documentation  and  implementation,  thereby  undermining  the  intended  purpose  of 
the  CMM. 


Managers  must  understand  the  SA-CMM  and  what  it  requires  in  resources  and  time  to 
achieve  a  certain  maturity  level.  Understanding  what  their  suppliers  have  gone  through  to 
achieve  certain  maturity  levels  of  the  SW-CMM  might  shed  some  light  on  this.  Data  is  cer¬ 
tainly  available. 


Another  misunderstanding  is  using  the  model  only  to  achieve  a  maturity  level  rating  rather 
than  to  instill  discipline  into  the  acquisition  processes.  This  attitude  may  reflect  a  quest  for 
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status  rather  than  a  legitimate  attempt  to  examine  business  needs  and  instill  process  discipline 
to  support  those  needs.  Managers  must  realize  that  CMMs  were  originally  developed  to  im¬ 
prove  processes,  not  to  achieve  maturity  levels.  The  SA-CMM  should  be  a  means  to  an  end, 
not  an  end  in  itself. 


3.1.6  Treat  Process  Improvement  as  a  Project 

Process  improvement  experts  agree,  treat  the  process  improvement  effort  as  a  well-run  pro¬ 
ject.  As  a  project,  develop  reasonable  plans  with  achievable  goals  based  on  business  needs; 
do  not  try  to  improve  everything  at  once;  obtain  long-term  sponsorship  and  commitments; 
devote  sufficient  resources  to  the  effort;  remember  to  plan  deployment  of  process  improve¬ 
ment  efforts  to  the  users;  use  process  improvement  planning  as  an  example  to  other  teams  as 
the  right  way  to  run  a  project. 


3.1.7  Generating  Policies,  Process  Descriptions,  and 
Procedures 

A  separate  process  organization  dedicated  to  the  development  and  deployment  of  key  proc¬ 
esses  will  greatly  enhance  the  chances  of  success.  The  use  of  such  a  dedicated  resource  will 
help  minimize  the  problem  of  process  improvement  competing  with  “real  work.”  However,  it 
is  important  that  project  staffs  be  involved  in  the  development  of  organization-level  processes 
early  on.  This  greatly  improves  communications  between  the  process  group  and  project 
staffs,  making  the  process  documentation  “real,”  and  thereby  improves  the  chance  of  having 
and  deploying  consistent  processes  across  projects. 


3.2  Applying  the  Principles 

There  were  a  number  of  important  elements  in  applying  the  above  principles  and  implement¬ 
ing  BSMO  process  improvement.  In  general,  however,  in  the  BSMO’s  process  improvement 
effort,  as  with  any  change  imposed  on  an  organization,  it  was  expected  that  there  would  be  an 
initial  decline  in  productivity  prior  to  achieving  the  desired  state  due  to  the  learning  curve, 
culture,  organizational  barriers,  etc.  This  well-known  concept  is  illustrated  in  Figure  1.  The 
idea  is  that  imposing  a  change  in  the  way  an  organization  operates  typically  results  in  a  de¬ 
crease  in  productivity.  After  a  transition  phase,  the  productivity  and  quality  of  work  product 
would  improve  to  a  desired  state.  During  the  transition  phase,  barriers  for  improvement  are 
overcome  by  such  approaches  as  training,  shift  of  cultural  perspectives  by  mandate,  and  bet¬ 
ter  application  of  technology.  In  the  BSMO,  the  decline  could  be  traced  to  deep  routed  cul¬ 
tural  issues,  conflicting  priorities,  unstable  management  hierarchy,  and  conflicting  govern¬ 
ment-contractor  roles.  In  addition,  BSMO  was  a  new  organization  that  engaged  people  from 
internal  and  external  organizations  who  came  with  their  own  way  of  doing  business,  their 
own  unique  terms,  management  styles,  and  ideas.  Eventually,  these  issues  were  ameliorated 
by  the  establishment  of  a  process  improvement  infrastructure,  serious  executive-level  com- 
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mitment  to  process  improvement,  and  constant  emphasis  on  training  in  project  management 
and  acquisition,  as  well  as  in  the  SA-CMM. 


Figure  1:  Managing  Organizational  Change 

Other  components  related  to  the  application  of  process  improvement  principles  are  described 
below,  categorized  according  to  how  the  principles  were  presented  in  Section  3.1. 


3.2.1  Stabilize  the  Environment  before  Attempting  Process 
Improvement 

Process  improvement,  whether  using  SA-CMM  or  not,  is  best  done  within  the  context  of  a 
stable  environment.  The  SA-CMM  was  used  to  help  the  BSMO  define  its  mission  and  resul¬ 
tant  responsibilities.  This  helped  stabilize  the  environment  for  improvement. 


Initially,  BSMO  personnel  were  confused  about  the  organization’s  vision,  strategic  goals, 
mission  and  functions,  and  concept  of  operations.  Interfaces  among  the  organization’s  ele¬ 
ments  and  suppliers  and  government  organizations  external  to  but  supporting  the  BSMO 
were  not  documented  and  were  often  personality  driven. 


Instability  within  the  BSMO  environment  relative  to  the  accomplishment  of  its  acquisition 
mission  resulted  in  numerous  starts  and  stops  and  redirection  not  only  of  the  process  im¬ 
provement  efforts  but  also  in  the  accomplishment  of  its  mission.  Acquisitions  were  not  pro¬ 
ceeding  well.  This  instability  can  be  attributed  to  many  causes,  but  stemmed  primarily  from 
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an  ever-changing  senior  executive  group  and  the  difficulty  in  settling  on  a  concept  of  opera¬ 
tions,  i.e.,  how  modernization  would  be  managed.  This  situation  was  exacerbated  by  the  lack 
of  articulated  vision,  strategic  goals,  and  documented  mission  and  functions.  The  struggle  in 
codifying  these  critical  areas  demonstrated  the  difficulty  in  understanding  the  organization’s 
vision  and  strategic  goals,  the  difficulty  in  translating  the  goals  and  objectives  into  a  viable 
organization,  and  the  subsequent  difficulty  in  creating  the  associated  documentation,  some¬ 
times  with  conflicting  results  that  did  not  clearly  articulate  the  BSMO’s  true  mission.  As  one 
example,  BSMO’s  initial  emphasis  was  to  use  the  “preferred”  contractor  to  do  virtually  eve¬ 
rything,  resulting  in  Government  personnel  abdicating  their  responsibility  to  manage  mod¬ 
ernization  effort.  Obviously,  this  shift  of  responsibility  and  workload  from  BSMO  personnel 
to  the  contractor  diluted  the  normal  Government  to  contractor  relationship  and  introduced 
risk  into  the  modernization  effort.  In  midst  of  this,  the  BSMO  was  attempting  to  codify  and 
improve  its  acquisition  management  processes. 


The  use  of  the  SA-CMM  provided  a  framework  and  a  way  out  of  the  confusion  caused  by  the 
poorly  understood  roles  and  responsibilities  (Government  to  contractor  relationship).  Devel¬ 
oping  and  employing  SA-CMM-consistent  organizational  directives  (policy  statements), 
process  descriptions,  and  associated  procedures  helped  stabilize  the  perturbations  due  to 
management  and  leadership  turnover  through  the  institutionalization  features.  While  there 
was  no  guarantee  this  institutionalization  would  solve  the  problem,  it  did  solidify  and  align 
the  roles  and  responsibilities  of  the  IRS  modernization  effort  with  the  true  mission  of  an  ac¬ 
quisition  organization.  It  also  promoted  the  indoctrination  of  new  managers,  causing  them  to 
rethink  their  previous  experience  in  terms  of  the  standard  processes  and  terminology  of 
BSMO. 

3.2.2  Leadership 

Earlier  BSMO  attempts  at  process  improvement  did  not  have  the  necessary  management  sup¬ 
port  to  become  firmly  entrenched.  Directives  (policy  statements),  process  descriptions,  and 
procedures  were  developed  previously  in  1999  but  used  only  sporadically  by  the  projects. 
Since  the  fall  of  2001,  increased  management  commitment  and  oversight  provided  the  neces¬ 
sary  focus  on  process  improvement,  resulting  in  much-needed  traction.  This  management 
commitment  was  particularly  evident  when  BSMO  came  under  increased  scrutiny  by  over¬ 
sight  organizations  (GAO,  Office  of  Management  and  Budget,  and  Treasury  Inspector  Gen¬ 
eral  for  Tax  Administration). 


3.2.3  Engaging  Experts  to  Help 

The  importance  of  engaging  experienced  process  improvement  and  acquisition  experts  to 
help  the  effort  cannot  be  over-emphasized.  The  BSMO  engaged  the  SEI  and  the  MITRE  Cor¬ 
poration,  both  federally  funded  research  and  development  centers  (FFRDCs)  throughout  this 
effort.  Typical  government  agencies  do  not  have  the  insight  or  broad  experience  that  the 
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FFRDCs  have  in  planning  process  improvements  and  applying  model-based  process  im¬ 
provements,  in  this  case  the  SA-CMM  model.  In  the  case  of  the  BSMO,  these  experts  came 
with  credentials  that  the  senior  IRS  executives  appreciated  and  listened  to.  The  guidance  the 
FFRDCs  provided  worked!  Many  of  the  recommendations  suggested  were  typical  of  most 
process  improvement  efforts  but  now  applied  to  acquisition  process. 


3.2.4  Avoid  “Big  Bangs” — Starting  Small 

The  BSMO  learned  from  the  failure  of  previous  process  improvement  attempts  to  not  try  to 
change  the  entire  organization  at  once.  This  is  a  typical  problem,  especially  when  managers 
do  not  understand  the  “reality”  of  process  improvement,  including  its  cost  and  duration.  Ini¬ 
tially,  the  BSMO  attempted  to  implement  a  set  of  standard  processes  across  every  project  in 
the  modernization  effort  simultaneously.  The  processes  were  not  sufficiently  matured  and  the 
Solutions  Acquisition  Process  Group  (SAPG)  did  not  have  the  capacity  to  support  so  many 
projects  all  at  once. 


After  the  baseline  review  in  January  2001,  the  review  team  recommended  that  the  BSMO 
target  a  much  smaller  subset  of  projects  to  work  with  initially.  Such  a  subset  would  be  more 
manageable,  changes  to  process  assets  could  be  better  controlled,  and  the  SAPG  could  pro¬ 
vide  the  necessary  support  to  the  earlier  adopters.  In  late  2001,  five  pilot  projects  were  identi¬ 
fied  and  the  SAPG  began  the  deployment  of  updated  processes  and  procedures  to  the  partici¬ 
pating  projects.  The  SAPG  also  provided  direct  assistance  to  them  in  their  process 
improvement  efforts. 


3.2.5  Use  the  SA-CMM  as  a  Starting  Point  (Applying  the 
Reference  Model  to  the  Organization) 

The  SA-CMM  is  designed  to  be  sufficiently  generic  for  use  by  any  government  or  industry 
organization,  regardless  of  its  size,  that  acquires  products.  When  applying  the  SA-CMM  to  a 
particular  organization,  translations  or  tailoring  may  be  needed  to  adapt  the  model  to  fit  the 
specific  organization.  These  translations  involve  mapping  the  model’s  generic  organization, 
language,  and  intent  to  how  the  organization  carries  out  its  business. 


In  addition,  the  model  terminology  has  been  made  as  generic  as  possible.  Model  terminology 
must  be  mapped  to  the  organization’s  terms.  Several  terms,  including  “partnership”  (with  the 
contractors),  “integrated  product  team,”  and  “matrix  management,”  had  been  inconsistently 
and,  in  some  cases,  incorrectly  applied  by  BSMO  personnel.  Mapping  the  SA-CMM  termi¬ 
nology  to  the  BSMO  way  of  doing  business  stabilized  terminology  and  its  concepts  of  opera¬ 
tions.  Table  1  shows  an  example  set  of  SA-CMM  terms  mapped  to  BSMO  concepts. 
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Table  1:  BSMO  Mapping  to  SA-CMM  Terms 


SA-CMM  Term 

IRS  BSMO  Term 

Software  Acquisition  Planning 

Solution  Acquisition  Planning 

Acquisition  Organization 

BSMO 

Contract 

Task  Order 

Project  Manager 

Acquisition  Project  Manager  (APM) 

Solicitation 

Task  Order  Development  And  Issuance 

Contract  Tracking  and  Oversight 

Task  Order  Monitoring 

Policy 

Directive 

The  BSMO,  in  fact,  used  the  SA-CMM  to  mean  “solution  acquisition”  CMM,  because  the 
BSMO  acquires  solutions,  not  just  software.  By  using  the  SA-CMM  as  a  tool  to  examine  the 
more  general  acquisition  processes  such  as  for  systems  and  services,  BSMO  has  stabilized  its 
processes  and  has  corrected  or  resolved  long-standing  management  issues. 

For  example,  the  term  “Solicitation”  in  the  SA-CMM  refers  to  a  process  for  the  preparation 
of  a  solicitation  package  such  as  a  request  for  proposal  (RFP),  soliciting  proposals  from  a 
number  of  contractors,  and  selecting  a  contractor  best  capable  of  satisfying  the  needs  of  the 
acquisition  organization.  In  BSMO  modernization,  the  selection  of  a  PRIME  contract  was 
accomplished  prior  to  the  process  improvement  effort.  Therefore,  in  the  current  BMSO  op¬ 
eration,  they  apply  the  SA-CMM  concepts  at  the  task  order  level,  considering  task  orders  as 
“mini-contracts.”  For  the  area  of  solicitation,  they  develop  and  issue  a  task  order,  receive  the 
contractor’s  proposal,  review  and  negotiate  the  task  order  terms  and  cost,  and  then  issue. 


Start  this  mapping  before  starting  to  document  organizational  processes.  Otherwise,  individu¬ 
als  involved  in  documenting  the  organizational  processes  and  procedures  will  have  different 
ideas  about  the  target  of  each  process  and  the  meaning  of  terms  and  concepts.  The  importance 
of  solidifying  the  application  of  the  model  and  the  organizational  terminology  cannot  be 
over-emphasized.  The  BSMO  terminology,  with  definitions,  is  now  documented  in  the 
BSMO  acquisition  directives. 


3.2.6  Treat  Process  Improvement  as  a  Project 

There  are  five  cornerstones  of  the  BSMO  process  improvement  project:  sponsorship,  the 
Management  Steering  Group,  the  Solutions  Acquisition  Process  Group,  Process  Action 
Teams,  and  BSMO  Process  Improvement  Planning. 

Sponsorship 

Sponsorship  is  shown  by  actions.  Senior  executives  not  only  stated  their  commitment  to  the 
SA-CMM-based  process  improvement  effort,  but  also  demonstrated  this  commitment  and 
leadership  by  their  ongoing  and  public  actions.  Management  made  it  clear  by  their  words, 
organizational  expectations,  and  commitment  that  process  improvement  was  a  critical  ele¬ 
ment  in  making  the  IRS  modernization  program  successful.  The  BSMO  staff  saw  senior  ex- 
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ecutives  using  every  tool  at  their  command  to  ensure  that  SA-CMM-based  process  improve¬ 
ment  took  hold  and  was  successful.  Resources  such  as  the  budget  for  direct  project  support, 
regular  training,  and  familiarization  sessions  were  provided.  The  senior  leadership  created, 
funded,  and  gave  responsibility  and  ample  authority  to  a  major  staff  element — the  Solutions 
Acquisition  Process  Group — devoted  to  process  improvement.  Sponsorship  was  especially 
important  in  managing  the  expectation  and  scrutiny  of  the  congressional  oversight  organiza¬ 
tions.  By  demonstrating  a  visible  and  tangible  commitment,  including  well  thought  out  and 
executed  process  improvement  plans  and  timely  appraisals,  BSMO  was  able  to  demonstrate 
to  the  oversight  organizations  that  the  IRS  had  a  serious  and  continuing  commitment  to  ac¬ 
quisition  process  improvement. 

Management  Steering  Group  (MSG) 

After  numerous  process  improvement  “marketing”  briefings  to  senior  IRS  officials,  it  was 
agreed  that  a  BMSO  Management  Steering  Group  (MSG)  would  be  created  to  lead  the  proc¬ 
ess  improvement  effort.  The  MSG  was  formally  chartered  in  September  2001  and  remains  the 
executive-level  driving  force  behind  process  improvement  today.  The  MSG  is  led  by  the  sen¬ 
ior  BSMO  executive  and  includes  all  IRS  senior  directors  responsible  for  the  various  compo¬ 
nents  of  the  IRS  modernization  program.  These  directors  also  are  in  charge  of  the  acquisition 
projects  that  are  the  target  groups  for  the  process  improvement  program.  Senior  executives 
realized  that  oversight  organizations  were  expecting  process  improvement  and  that  without 
such  an  effort  modernization  funding  could  potentially  be  reduced  or  eliminated.  This  be¬ 
came  the  initial  “business  case”  for  BSMO  process  improvement. 

Solutions  Acquisition  Process  Group 

The  Solutions  Acquisition  Process  Group  (SAPG)  was  formally  chartered  by  the  MSG  in 
September  2001.  The  SAPG  leads  the  day  to  day  process  improvement  effort  and  is  the  agent 
of  change  for  the  organization.  The  SAPG  leads  the  effort  in  documenting  BSMO  standard 
processes  and  procedures,  deploying  these  assets  to  the  modernization  project  teams  and  sup¬ 
porting  their  implementation  through  training  sessions  and  direct  project  support.  The  SAPG 
reports  regularly  to  the  MSG  and  also  arranges  for  formal  appraisals  and  reviews. 

Process  Action  Teams 

The  SAPG  charters  each  process  action  team  (PAT),  assigns  a  team  lead,  and  expects  a 
documented  plan  and  schedule  for  the  PAT  efforts  in  process  improvement.  In  BSMO,  PATs 
typically  implement  short-term  efforts  that  have  a  specific  goal  and  vision.  The  SAPG  sup¬ 
ports  the  PATs  by  providing  guidance,  examples,  and  in  many  cases  resources  to  staff  and 
lead  the  PATs. 

BSMO  Process  Improvement  Planning 

Formal  planning  of  process  improvement  activities  and  appropriate  executive  review  is  es¬ 
sential  for  success.  Early  in  the  process  improvement  effort,  the  MSG  and  SAPG  established 
realistic  goals  and  defined  and  documented  meaningful  and  actionable  process  improvement 
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activities.  This  served  to  demonstrate  to  the  modernization  project  participants  the  impor¬ 
tance  and  value  of  planning  activities  and  measuring  performance  against  the  plan  and  sched¬ 
ule — just  the  behavior  the  SA-CMM  model  requires. 


The  leading  tenet  of  the  effort  is  to  treat  process  improvement  as  a  project  and  to  approach 
and  manage  it  in  a  disciplined  way.  A  three-year  BSMO  Process  Improvement  Strategic  Plan 
was  developed  and  approved  by  the  MSG.  Specific  initiatives  associated  with  each  goal  were 
documented  in  the  Strategic  Plan.  Once  the  plan  was  approved,  the  SAPG  developed  a  de¬ 
tailed  Process  Improvement  Action  Plan  (PIAP)  for  the  current  year.  At  the  end  of  each  year, 
the  Strategic  Plan  is  “rolled”  forward  one  year  and  the  PIAP  developed  for  the  new  year. 
Figure  2  shows  the  interrelationships  of  the  components  to  this  planning  process.  (Note  that 
the  appraisal  results  feed  the  planning  process.  See  the  discussion  on  appraisals  below.) 


BSMO  PI 
Strategic  Plan 
Contains  goals  and 


BSMO 

Goals 


Detailed  assignments 


Lessons 

Learned 


SCE 

Results 


Figure  2:  BSMO  PI  Planning 

The  planning  process  is  iterative  and  continuous.  The  BSMO  SAPG  has  developed  plans  in 
consultation  with  practitioners  on  the  teams,  using  the  SA-CMM  and  the  SEI  IDEAL2  ap¬ 
proach  as  a  framework.  Planning  is  used  as  part  of  an  active  feedback  mechanism,  generating 
specific  activities  and  steps  that  can  be  assigned,  executed,  and  measured. 

Adding  the  planning  component  forced  IRS  personnel  to  have  realistic  expectations  about 
productivity  and  schedule.  When  developing  schedules,  they  engaged  and  listened  to  the  ex¬ 
perts,  both  within  the  IRS  and  the  PRIME  Alliance.  As  a  result,  their  process  improvement 
schedules  were  based  on  realistic  estimates  and  enabled  the  organization  to  accept  the  sched¬ 
ules. 


2  Initiating,  Diagnosing,  Establishing,  Acting,  &  Learning.  See  <http://www.sei.cmu.edu/ideaF>. 
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BSMO  Process  Improvement  Strategic  Plan 


The  Strategic  Plan  was  developed  by  the  SAPG  for  MSG  approval.  The  plan  lays  out  process 
improvement  goals,  objectives,  and  initiatives  for  the  next  three  years  and  is  updated  on  a 
yearly  basis.  It  ties  the  process  improvement  goals  to  organizational  goals;  defines  the  proc¬ 
ess  improvement  strategy,  yearly  goals,  and  initiatives;  and  addresses  resources,  risks,  roles, 
and  responsibilities.  The  plan  was  built  on  the  SEI  IDEAL  approach.  It  is  the  cornerstone 
planning  document  for  the  process  improvement  effort  and  also  serves  as  a  communication 
vehicle  to  senior  management  and  oversight  organizations. 


All  members  of  the  MSG  signed  the  initial  BSMO  Process  Improvement  Strategic  Plan  in 
March  2002,  which  set  a  course  for  increased  attention,  definitive  goals,  and  necessary  activi¬ 
ties  to  improve  the  acquisition  capability  of  the  IRS. 

Process  Improvement  Action  Plans 

Based  on  the  approved  Process  Improvement  Strategic  Plan,  a  Process  Improvement  Action 
Plan  (PIAP)  was  written  by  the  SAPG  for  the  upcoming  year’s  activities.  The  PIAP  is  the 
tactical  plan  and  is  more  specific.  A  set  of  well  thought  out  initiatives  were  developed  that  all 
mapped  back  to  the  strategic  goals  for  the  year.  For  example,  the  PIAP  for  2003  included  the 
definition  of  13  initiatives.  The  plan  specified  each  initiative’s  title,  description,  effort,  sched¬ 
ule,  priority,  and  responsible  office  or  individual.  The  PIAP  was  also  approved  by  the  MSG 
and  assigned  back  to  the  SAPG  for  implementation. 

Detailed  Work  Plans 

For  each  approved  initiative,  the  responsible  person  or  office  developed  a  detailed  work  plan 
that  described  the  work  breakdown  structure  (WBS)  for  everything  involved  in  completing 
the  initiative.  These  work  plans  were  approved  and  tracked  by  the  SAPG  chairman  and 
“statused”  at  each  SAPG  meeting.  These  work  plans  served  two  purposes.  First,  they  were  a 
way  for  the  individuals  assigned  to  organize  their  work  to  communicate  to  the  organization 
the  who,  what,  why,  when,  and  benefit  of  the  initiative.  But  more  importantly,  they  demon¬ 
strated  to  BSMO  as  a  whole  the  benefit  of  “planning  your  work,  and  then  working  your 
plan.” 


3.2.7  Generating  Policies,  Process  Descriptions,  and 
Procedures 

The  BMSO  Acquisition  Management  policies,  processes,  and  descriptions  lay  out  the  tasks, 
activities,  and  steps  associated  with  developing,  issuing,  monitoring,  and  closing  out  task  or¬ 
ders. 

Initial  thinking  was  that  a  large  group  split  up  into  smaller  groups  could  generate  the  required 
policies,  process  descriptions,  and  procedures  concurrently.  Each  group  would  work  on  a  key 
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process  area  (KPA)  from  the  SA-CMM,  writing  the  necessary  policy,  processes,  and  proce¬ 
dures,  and  then  the  groups  would  come  together  and  integrate  the  pieces — all  without  engag¬ 
ing  the  practitioners  who  actually  manage  the  acquisition  contract.  This  approach  failed.  The 
processes  were  not  written  with  the  same  voice,  they  did  not  have  the  benefit  of  reality,  and, 
when  published,  no  one  followed  them  nor,  in  some  cases,  knew  about  them.  And  the  termi¬ 
nology  was  not  consistent  throughout  the  first  set  of  assets. 


The  BSMO  set  up  a  system  of  teams  to  work  on  the  different  processes  thought  to  be  the  fu¬ 
ture  way  the  BSMO  would  operate.  The  teams  consisted  of  subject  matter  experts  (SMEs) 
from  various  disciplines  and  with  differing  skills  and  experiences.  With  these  various  back¬ 
grounds  and  no  clear  direction  on  the  BSMO’s  vision  and  goals  and  concept  of  operations 
(CONOPS),  a  “I’ve  always  done  it  this  way”  mentality  was  noticeable  in  many  of  the  process 
improvement  artifacts.  The  basic  intent  was  good,  that  is,  bringing  in  the  SMEs  who  could 
describe  what  processes  had  to  be  created  and  installed.  However,  the  loyalty  of  those  indi¬ 
viduals  typically  was  to  their  home  office  and  the  way  they  did  business  and  not  in  establish¬ 
ing  processes  for  a  new  organization.  This  initial  set  of  process  assets  were  not  well  balanced, 
nor  were  they  integrated  vertically  or  horizontally.  This  situation  was  exacerbated  by  the 
managers  of  these  external  offices  being  rivals  of  the  BSMO  organization  itself. 


After  the  process  improvement  began  in  earnest,  the  BSMO  SAPG  established  a  key  PAT  as  a 
“core  writing  group”  that  included  representatives  from  the  acquisition  projects.  This  core 
writing  group  settled  on  the  fundamental  concepts  of  how  to  apply  the  model  to  BMSO,  set¬ 
tled  on  terminology,  and  documented  what  each  acquisition  project  team  did  for  process  areas 
described  by  the  model.  When  they  were  done  documenting  each  process,  the  core  writing 
team  compared  the  documented  process  with  the  SA-CMM  model  and  made  adjustments. 
Additional  activities  were  engineered  into  the  processes  and  deployed  to  the  acquisition 
teams.  Because  the  core  writing  group  did  one  KPA  at  a  time,  this  method  of  documenting 
took  approximately  15  calendar  months  for  all  required  KPAs,  but  the  result  was  a  solid  set  of 
processes  and  procedures  that  BSMO  acquisition  teams  could  understand  and  follow. 


3.2.8  Deployment 

Initially,  the  BSMO  attempted  to  deploy  a  set  of  standard  processes  across  every  project  in 
modernization  simultaneously.  This  approach  did  not  work — there  were  too  many  projects. 
Furthermore,  the  SAPG  did  not  have  the  capacity  to  support,  communicate,  and  “socialize” 
these  processes  to  the  entire  organization.  Here  we  discuss  the  role  of  training,  communica¬ 
tions,  and  appraisals  in  support  of  the  deployment  processes. 

Training 

The  BSMO  process  improvement  team  learned  not  to  assume  that  everyone  has  the  same  un¬ 
derstanding  of  how  the  organization  works.  Particularly  in  a  relatively  new  organization, 
people  use  different  terms  and  different  unwritten  processes  and  have  different  ideas  as  to 


16 


CMU/SEI-2004-TR-002 


who  does  what.  It  was  realized  that  a  significant  amount  of  training  would  be  needed  to  en¬ 
sure  that  individuals  understood  what  the  process  improvement  effort  was  about. 


The  BSMO  developed  training  modules  (one-to-two-hour  briefings)  for  each  of  the  BSMO 
process  areas.  Sets  of  these  briefings  are  spread  over  a  period  of  four  to  five  weeks  and  are 
repeated  approximately  three  times  per  year.  In  addition,  a  two-and-a-half-day  SA-CMM 
class  is  offered  approximately  three  times  per  year.  As  in  any  organization,  IRS  acquisition 
processes  touch  other  processes  that  are  not  directly  tied  to  acquisition,  such  as  configuration 
management,  program  control,  and  system  testing.  The  BSMO  has  tried  to  ensure  that  a  com¬ 
plete  picture  of  these  interrelationships  is  presented  to  the  attendees  so  they  are  well  under¬ 
stood. 

Communication  is  at  the  Heart  of  Process  Improvement 

The  January  2001  baseline  review  identified  communication  as  a  problematic  area  for  the 
BSMO.  Even  though  the  processes  documented  by  the  core  writing  group  were  placed  on  the 
process  asset  library,  few  if  any  of  the  BSMO  project  teams  at  that  time  knew  what  they  were 
and  what  management  expectations  were  regarding  them.  In  order  to  improve  communica¬ 
tions,  the  BSMO  SAPG  implemented  a  communications  plan  that  is  based  on  the  goals  and 
objectives  of  the  BSMO  Process  Improvement  Strategic  Plan  and  the  current  year’s  PIAP. 
BSMO  personnel  also  hold  biweekly  meetings  with  professional  communications  staff.  Cur¬ 
rent  plans  have  specific  goals  based  on  the  following  parameters:  audience,  stage  of  commu¬ 
nication,  triggering  mechanism,  “messages,”  the  vehicle,  and  the  delivery  method.  Voice 
mails,  flyers,  presentations,  e-mails,  weekly  “snippets,”  posters,  and  key  messages  for  pro¬ 
gram  executives  are  developed  and  deployed  throughout  the  year. 

Appraisals  are  Important 

Periodic  assessments  and  appraisals  of  the  project,  both  internal  and  external,  are  absolutely 
essential  to  baseline  the  current  state  of  the  organization’s  process  capability  and  to  keep  the 
necessary  focus  on  process  improvement,  especially  the  acquisition  projects’  perspectives. 


The  BSMO  conducted  a  “baseline  review”  of  the  BSMO  process  capability  in  January  2001 
using  the  SA-CMM  as  the  reference  model.  This  review  was  considered  an  informal  assess¬ 
ment  led  by  the  SEI,  with  the  bulk  of  the  review  team  being  IRS  and  MITRE.  The  team  in¬ 
terviewed  the  modernization  project  managers,  the  program  directors,  and  other  functional 
area  personnel  (e.g.,  procurement,  program  control).  Although  not  a  full  formal  assessment, 
this  review  was  critical  in  establishing  the  foundation  for  process  improvement.  It  provided 
BSMO  senior  management  a  first  glimpse  at  where  they  were  in  having  a  disciplined  ap¬ 
proach  to  acquisition. 


Since  the  January  2001  baseline  review,  the  BSMO  has  undergone  three  additional  re¬ 
views/appraisals  in  its  process  improvement  journey. 
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The  group  of  five  pilot  projects  selected  for  process  improvement  underwent  an  assessment 
(CBA-IPI3)  led  by  the  SEI  in  June  2002.  Most  model  goals  (SA-CMM  level  2  KPA  goals) 
were  satisfied,  but  there  were  a  number  of  improvement  opportunities  identified.  There  were 
enough  positive  activities  being  performed  in  the  BSMO  that  the  improvement  opportunities 
seemed  very  doable  to  the  group,  and  they  felt  that  they  were  on  the  right  track. 


After  significant  effort  and  focus  on  the  identified  improvement  opportunities,  the  same  pro¬ 
ject  group  underwent  a  formal  Software  Capability  Evaluation  (SCE)  in  December  2002.  In 
the  final  briefing  on  December  6,  2002,  the  SCE  team  reported  that  this  group  was  operating 
at  maturity  level  2  of  the  SA-CMM.  This  was  the  first  civilian  government  group  of  multisys¬ 
tem  projects  ever  to  achieve  this  recognition.  The  BSMO  had  committed  to  their  oversight 
organizations  to  undergo  an  evaluation,  with  the  goal  of  achieving  level  2.  They  did  it  on  time 
and  according  to  plan. 


After  the  SCE,  the  Strategic  Plan  and  Process  Improvement  Action  Plan  were  updated,  add¬ 
ing  other  modernization  projects  to  the  process  improvement  activities  for  fiscal  year  2003. 


After  many  training  sessions  and  direct  project  support,  the  BMSO  underwent  a  baseline  re¬ 
view  in  October  2003.  Ten  projects  participated.  This  review  was  important  because  the  or¬ 
ganization  needed  to  determine  where  it  stood  in  the  full-scale  rollout,  as  well  as  when  the 
next  formal  appraisal  should  be  planned.  The  review  presented  a  number  of  improvement 
opportunities  for  all  these  projects  and  recommended  that  the  next  formal  evaluation  should 
take  place  in  September  2004. 


3  CMM-Based  Appraisal  for  Internal  Process  Improvement 
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4  Results 


Since  the  first  baseline  review  in  January  2001,  process  improvement  in  the  BSMO  has 
gained  significant  traction,  and  the  motivation  for  process  improvement  has  now  shifted  to 
business  benefit.  The  BSMO  is  now  realizing  the  benefit  of  documenting  and  improving  or¬ 
ganizational  processes  and  using  them  in  daily  work.  The  BSMO  now  knows  it  makes  good 
business  sense  that 

•  When  leadership  changes  there  is  an  underlying  documented  acquisition  process  to  guide 
the  organization. 

•  Internal  project  “thrashing,”  i.e.,  constant  questions,  false  starts,  and  rework,  is  mini¬ 
mized  because  the  project  is  organized  better,  has  a  documented  plan  and  schedule,  and 
the  communication  is  free  flowing  and  accessible. 

•  Training  can  actually  be  based  on  the  organizational  processes,  because  they  are  docu¬ 
mented.  The  BSMO  no  longer  has  to  depend  on  generic  training  that  has  little  applicabil¬ 
ity  to  the  work  being  done. 

•  Planning  ahead  allows  foresight  and  predictability  of  issues  that  can  be  dealt  with  early, 
when  they  are  easier  and  less  costly  to  address. 

•  New-hires  now  have  documented  organizational  processes  and  project  plans  to  go  to. 

•  No  longer  is  90%  of  the  work  done  by  10%  of  the  work  force.  Planning  ahead  helps  dis¬ 
tribute  the  work  load  to  all  team  members,  making  everyone  feel  a  part  of  the  team. 

•  Contractor  and  government  roles  are  well  understood  and  documented  up  front. 


Activities  are  now  planned  in  advance  in  the  acquisition  management  plan  for  each  task  or¬ 
der.  Planning  ahead  and  having  a  systematic  method  based  on  documented  processes  allows 
acquisition  project  managers  to  manage  more  methodologically;  risks  are  anticipated,  rework 
is  minimized,  meetings  are  more  on  target,  progress  is  made. 

Result  1:  Stabilization  of  Terminology 

The  terms  that  have  been  solidified  and  documented  in  the  organizational  processes  are  now 
in  common  use,  such  as  acquisition  project  manager,  acquisition  management  plan,  artifacts, 
and  repository  (document  management  system).  The  terms  used  for  the  process  areas  are  also 
now  in  the  daily  BSMO  vocabulary:  Task  Order  Development  and  Issuance,  Task  Order 
Monitoring,  Transition  to  Support,  etc. 
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Result  2:  Silver  Bullet? 


Even  with  the  application  of  SA-CMM  concepts,  the  IRS  modernization  program  continues 
to  face  significant  challenges,  as  is  typical  with  large,  complex  modernization  efforts.  While 
the  SA-CMM  as  a  management  model  was  necessary  to  improve  the  BSMO’s  management 
capability,  it  is  not  a  silver  bullet.  It  does  not  guarantee  program  performance;  rather,  it  sets 
the  foundation  for  improving  acquisition  management  performance. 


It  should  be  noted  that,  like  other  process  improvement  models,  the  SA-CMM  does  not  pre¬ 
scribe  “how”  to  establish  true  executive  sponsorship  and  commitment.  This  is  something  the 
BSMO  had  to  develop  itself.  In  the  end,  both  the  acquiring  organization  (the  BSMO)  and  the 
providing  organization  (CSC  PRIME  Alliance)  have  had  to  adopt  cultures  of  communication, 
cooperation,  and  collaboration  and  be  open  to  understanding  and  evolving  their  processes. 

Result  3:  Training 

Based  on  the  standard  organizational  processes  and  procedures,  there  are  now  training  ses¬ 
sions  that  are  available  to  assigned  acquisition  project  managers,  staff,  and  new-hires.  These 
have  helped  to  flatten  the  organizational  learning  curve. 

Result  4:  Executive  Expectations 

Senior  leadership  has  embraced  the  CMM  concepts  that  are  embodied  in  the  organizational 
processes  and  expect  their  acquisition  project  managers  and  staff  to  adhere  to  them.  As  such, 
fundamental  project  management  activities  are  beginning  to  be  woven  into  the  organizational 
culture  and  day  to  day  activities. 

Result  5:  Measuring  and  Improving  Processes 

The  standard  documented  BSMO  processes  have  helped  the  IRS  procurement  organization  in 
performing  root-cause  analyses  of  the  part  of  the  acquisition  process  they  are  responsible  for. 
These  analyses  helped  to  characterize  the  pre-award  timeline  of  the  solicitation  process  and  to 
define  the  measures  needed  improve  this  process,  as  well  as  form  the  basis  for  future  acquisi¬ 
tions. 

Results  6:  Repository  and  Its  Implications 

The  BSMO  has  overseen  the  establishment  and  growth  of  a  program  repository  (document 
management  system)  in  which  people  can  submit,  update,  check  out,  and  control  their  acqui¬ 
sition  assets.  Currently  there  are  over  18,000  files  stored  in  the  repository.  This  repository  has 
improved  communication  and  coordination  among  teammates  and  has  enabled  easier  re¬ 
trieval  of  artifacts  for  assessment  purposes.  No  longer  do  team  members  need  to  seek  out  the 
acquisition  project  manager  or  even  the  contractor  for  important  documentation.  There  are 
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areas  set  aside  in  the  repository  for  every  project  on  the  modernization  effort,  and  everyone 
with  the  requisite  permission  has  access  to  it. 


Result  7:  Alignment  with  Contractors 

The  BSMO  staff  learned  that  the  acquiring  organization’s  team  and  its  activities  is  a  project 
itself  and  should  be  planned,  managed,  and  run  as  such.  They  also  expect  their  contractor 
team  to  plan  and  manage  itself.  By  working  in  a  collaborative  learning  environment  that  fos¬ 
ters  mutual  trust  and  dispenses  as  much  as  possible  with  gratuitous  recriminations,  both  par¬ 
ties  have  found  fruitful  common  ground  for  success.  Using  the  SA-CMM  as  the  basis  for  or¬ 
ganizational  process  improvement  has  provided  them  a  common  foundation  that  has 
promoted  better  understanding  of  roles,  responsibilities,  and  alignment  of  processes.  BSMO’s 
PRIME  contractor  is  also  in  effect  a  solutions  acquisition  organization  (in  addition  to  being  a 
systems  integrator),  which  further  illustrates  the  need  for  both  organizations  to  adopt  the 
same  framework,  such  as  SA-CMM. 

Result  8:  Policies,  Processes,  and  Procedures 

Processes,  procedures,  policies,  and  other  work  rules  were  clearly  thought  out  and  docu¬ 
mented  to  enable  each  party  to  the  contract  between  providing  and  acquiring  organizations  to 
understand  what  is  expected  of  their  project  teams  and  to  make  sure  that  necessary  resources 
are  in  place. 

Result  9:  Planning  and  Use  of  Plans 

Acquisition  team  members  now  plan  out  activities  for  each  task  order  in  advance  in  the  ac¬ 
quisition  management  plan.  Roles,  responsibilities,  and  expectations  are  more  clearly  under¬ 
stood  and  team  members  feel  more  involved  in  the  program — it’s  not  just  “the  contractor’s 
problem”  any  more. 


Organizations  (procurement,  the  BSMO,  and  the  business  owners)  are  better  aligned  and 
have  a  common  foundation  for  understanding  and  process  improvement. 


The  BSMO  has  realized  a  better  working  relationship  with  its  procurement  organization  by 
having  standard  acquisition  processes  and  procedures.  They  don’t  have  to  start  with  a  blank 
piece  of  paper  in  their  process  discussions.  They  can  focus  on  a  documented  step  or  series  of 
procedures  and  get  to  the  heart  of  the  issue  for  resolution  much  faster. 

Result  10:  Transition  from  IT  Culture 

The  culture  of  the  IRS  went  from  that  of  being  a  developer  to  being  an  acquisition  organiza¬ 
tion.  The  IRS  realized  that  they  did  not  have  the  capacity  or  capability  to  develop  and  deploy 
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modernized  systems  themselves.  They  had  to  transition  and  let  a  large  set  of  expert  compa¬ 
nies  do  the  development  and  integration  for  them. 


Result  11:  Better  Management  Skills 

The  BSMO  developed  and  institutionalized  better  management  skills.  They  did  this  through 

•  the  use  of  more  disciplined  approaches,  assessments,  techniques,  and  acquisition  man¬ 
agement  planning 

•  the  use  of  tools  such  as  earned  value,  management  information  center,  integrated  master 
schedule,  and  performance  measurement 

•  the  “simmering”  value  of  time  and  experience 

As  a  result,  their  managers  on  the  program  have  a  better  handle  on  modernization.  They  find 
they  can  spend  their  time  managing  and  not  micromanaging.  They  are  doing  the  work  they 
need  to  do  and  leaving  the  work  of  development,  testing,  and  integration  where  it  belongs- 
with  the  providing  organization. 

Result  12:  Empowerment 

People  in  the  BSMO  feel  empowered  because  the  organization  is  now  clear  on  who’s  respon¬ 
sible  for  what.  The  SA-CMM  has  helped  them  determine  this.  The  BSMO  provides  each  ac¬ 
quisition  project  manager  (APM)  with  a  certificate  that  clearly  specifies  the  responsibilities 
of  the  role  they  fill.  The  APMs  are  held  to  this  standard  by  the  BSMO.  The  BSMO  has  a  gov¬ 
ernance  structure  that  describes  the  cost,  schedule,  and  commitment  thresholds  that  guide 
each  APM  in  their  decision-making  processes.  Also,  roles  and  responsibilities  for  everyone 
on  each  project  team  are  defined  in  their  project’s  acquisition  management  plan. 

Result  13:  Appraisal  Results 

An  appraisal  in  December  2002  rated  the  BSMO  Core  Modernization  Group  of  the  five  core 
modernization  projects  as  operating  with  a  process  capability  of  SA-CMM  maturity  level  2. 
The  BSMO  is  the  first  federal  civilian  agency  group  to  be  evaluated  at  this  level. 

Result  14:  Resources 

One  of  the  benefits  of  CMM  appraisals  is  that  they  formally  bring  up  issues  that  may  other¬ 
wise  fall  on  deaf  ears.  One  of  these  for  BSMO  was  the  lack  of  sufficient  resources  to  manage 
modernization.  Since  this  issue  was  raised  with  no  attribution  of  source,  many  additional  per¬ 
sonnel  have  been  hired,  with  skills  more  in  line  with  the  role  of  the  acquisition  organization. 
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Result  15:  Organizational  Pride 

The  successes  the  BSMO  has  had  in  process  improvement  have  been  a  source  of  great  pride 
for  their  organization.  Because  they  were  the  first  federal  civilian  agency  group  to  achieve 
SA-CMM  level  2,  other  government  agencies  have  come  to  them  to  find  out  how  they  did  it. 
Senior  IRS  officials  presented  the  participating  projects  and  groups  with  awards,  ceremonies, 
and  a  recognition  luncheon  for  this  accomplishment. 

Result  16:  Oversight  Scrutiny 

The  BSMO’s  oversight  groups  (GAO  and  Department  of  Treasury  Inspector  General’s  office) 
have  gained  confidence  in  the  BSMO’s  organizational  maturity  because  the  BSMO  has  been 
recognized  as  having  successfully  applied  CMM  level  2  concepts  on  modernization.  Quoting 
a  June  2003  GAO  report:  “IRS  made  significant  progress  in  improving  its  modernization 
management  controls. .  .Significant  among  these  efforts  were  IRS’  achievements  in  improving 
its  software  acquisition  practices.” 
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5  Summary 


5.1  Lessons  Learned 

The  following  discussions  reflect  some  of  the  lessons  learned  during  the  process  improve¬ 
ment  effort.  Many  of  these  are  focused  not  explicitly  on  process  improvement  but  more  on 
the  underlying  problems  surfaced  during  the  process  improvement  work,  and  specifically 
with  the  organization  itself,  e.g.,  vision,  mission,  and  management.  Many  of  the  lessons 
learned  are  inextricably  linked,  each  affecting  others. 

Senior  Management  Must  Lead  the  Organization’s  Cultural 
Shift  to  Meet  Vision  and  Strategic  Goals 

It  isn’t  enough  that  executives  hand  down  pronouncements  or  write  memos  to  the  staff.  It 
takes  leadership. 


The  following  paraphrasing  from  Software  Runaways  [Glass  98]  originally  fit  the  situation  at 
the  BSMO: 

“The  serious  delays  in  the  [BSMO process  improvement  effort]  were  rooted  in 
deep  cultural,  political,  and  organizational  problems  within  that  agency,  many 
observers  agree.  ’’ 

The  recommendation  may  be  translated  into  establishing  and  articulating  at  least  a  ten¬ 
tative  organization,  mission  and  function,  and  CONOPS  prior  to  starting  process  im¬ 
provements  efforts.  This  would  set  the  foundation  and  needed  stability  for  process  im¬ 
provement.  Further,  the  group  responsible  for  coordinating  process  improvement 
efforts  must  be  sponsored,  empowered,  chartered,  and  supported  by  management  to 
help  management  lead  the  cultural  shift. 

Leadership 

Without  the  leadership  at  the  highest  levels  of  management,  this  effort  would  not  have  been 
successful.  Considering  the  path  of  deploying  results  to  other  BSMO  projects,  this  strong 
leadership  must  be  maintained. 
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Initially,  BSMO  managers  making  strategic  decisions  were  challenged  in  most  aspects  of  ac¬ 
quisition  project  management,  such  as  “setting  up”  an  organization,  codifying  its  mission  and 
functions,  and  resolving  external  interface  and  political  battles. 


Assign  skilled  and  experienced  managers  and  staff  at  all  levels  that  are  committed  to 
and  expected  to  follow  organizational  processes. 

Interpret  the  Model  in  Light  of  Business  or  Mission  Needs 

Effective  and  efficient  acquisition  processes  are  critical  to  successful  process  improvement, 
but  the  quality  of  their  output  can  only  be  determined  in  the  context  of  the  business  needs  of 
the  organization.  The  SA-CMM  should  be  interpreted  in  the  context  of  the  business  or  mis¬ 
sion  needs  of  the  organization.  Key  concepts  as  to  how  the  SA-CMM  must  be  tailored  or  ap¬ 
plied  to  the  organization  must  be  clearly  understood  and  agreed  to  up  front. 


Working  with  experts,  the  BSMO  successfully  interpreted  the  SA-CMM  in  terms  of  its  busi¬ 
ness  needs  to  make  the  BSMO  efforts  responsible,  accountable,  and  predictable.  Note  that 
demonstrated  expertise  in  the  SA-CMM  and  process  improvement  is  needed — not  self  pro¬ 
claimed  expertise. 

Prematurely  Mandating  Achievement  of  a  Specific  CMM  Level 
Leads  to  Failure 

Arbitrary  imposition  of  a  CMM  timeline  encourages  organizational  shortcuts  in  process 
documentation  and  implementation,  thereby  undermining  the  intended  purpose  of  the  CMMs. 
Managers  must  understand  the  SA-CMM  and  what  it  takes  in  resources  and  time  to  achieve  a 
certain  maturity  level. 


Another  indication  of  a  similar  misunderstanding  is  using  the  model  only  to  achieve  a  matur¬ 
ity  level  rating,  rather  than  instilling  discipline  into  the  process.  This  attitude  may  reflect  a 
quest  for  status  rather  than  a  legitimate  attempt  to  examine  business  needs  and  instill  process 
discipline  to  support  these  needs.  The  SA-CMM  should  be  a  means  to  an  end,  not  an  end  in 
itself. 


Initially  BSMO  management  took  the  “we  will  be  level  2  in  six  months”  approach.  This 
doomed  the  effort  from  the  beginning.  It  was  not  until  the  management  realized  all  the  factors 
that  lead  to  process  improvement  and  set  out  a  “reasonable”  plan  and  managed  to  the  plan 
that  the  process  improvement  effort  was  successful. 
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SAPG  and  Process  Descriptions 

Early  involvement  of  project  staff  in  the  development  of  the  organization-level  processes 
greatly  improves  communications  between  the  process  improvement  teams  and  the  acquisi¬ 
tion  projects,  and  improves  the  chances  of  having  consistent  processes  across  projects.  This 
involvement  with  project  staff  was  and  is  one  of  the  cornerstones  of  the  BSMO  process  im¬ 
provement  approach.  The  approach  involved 

•  establishing  a  core  SAPG  and  using  quick-hitting  PATs  to  tackle  documenting  the  proc¬ 
esses,  i.e.,  having  a  core  set  of  people  write  the  artifacts,  then  having  subject  matter  ex¬ 
perts  or  stakeholders  from  the  affected  areas  review  the  documents.  In  this  way  all  the  ar¬ 
tifacts  can  have  a  consistent  thread. 

•  working  with  practitioners  on  their  high-priority  process  needs  and  satisfying  them 
quickly 

•  ensuring  that  it’s  clear  who  is  in  charge  of  the  process  artifacts 

Roles  and  Responsibilities 

The  SA-CMM  was  used  in  creating  opportunities  for  sharing  and  collaboration.  It  provides 
an  excellent  framework  to  improve  chances  of  success  in  a  large  and  complex  program.  It 
provides  formal  terms  that  can  be  used  (or  translated,  as  the  BSMO  has  done)  to  align  the 
understandings  and  communications  between  the  acquiring  and  providing  organizations.  And 
it  provides  an  excellent  model  to  assist  in  developing  program  management  and  planning  best 
practices. 


5.2  In  the  End 

As  of  early  2004,  the  IRS  modernization  effort  has  delivered  a  number  of  modernization  ca¬ 
pabilities  that  provide  tangible  benefits  to  taxpayers  and  improve  the  efficiency  and  effec¬ 
tiveness  of  the  tax  administration  systems,  such  as  Where  s  My  Refund?,  Where  s  My  Advance 
Child  Tax  Credit?,  Internet  EIN,  Modernized  Electronic  Filing,  and  a  new  internal  Human 
Resources  management  system.  Throughout  the  journey,  the  BSMO  Process  Improvement 
Program  has  been  a  significant  contributor  in  these  successes  and  will  continue  to  execute  its 
process  improvement  role  as  modernization  moves  forward. 


In  the  end,  the  BSMO  found  that,  while  process  improvement  is  not  a  silver  bullet  and  it  is 
not  the  only  arrow  needed  in  management’s  quiver,  they  needed  stable  and  institutionalized 
processes  to  begin  to  impact  improved  organizational  performance.  The  institutionalization 
of  process  and  process  improvement  into  the  culture  of  their  team  was  vital  to  the  overall 
success  they  were  striving  for. 
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However,  the  “real”  benefits  are  realized  in  the  journey,  rather  than  in  the  end  result  of  a  ma¬ 
turity  level.  Along  the  way  more  discipline,  awareness,  and  skilled/trained  people  helped 
identify  resource  needs,  got  task  orders  more  timely,  and  identified  supplier  problems  more 
easily.  Morale  is  high,  people  are  proud,  and  they  see  the  light  at  the  end  of  the  tunnel. 


This  entire  process  (and  BSMO  is  not  stopping  here)  basically  allowed  the  IRS  to  see  where 
the  problems  were  in  a  systemic  way  instead  of  “groping”  and  putting  out  fires.  Planning 
ahead  and  having  systematic  methods  and  associated  tools  allows  acquisition  project  manag¬ 
ers  to  manage  more  methodologically.  Meetings  are  more  on  target,  progress  is  made,  and 
there  is  less  unproductive  activity. 


Having  succeeded  in  attaining  SA-CMM  maturity  level  2  on  the  initial  five  projects,  the 
BSMO  is  currently  working  to  sustain  their  maturity  with  those  five  projects  and  also  to  de¬ 
ploy  processes  to  the  remaining  modernization  projects. 


Not  everything  they  have  done  applies  to  every  other  program  involved  in  acquisition.  Their 
general  approach  to  understanding,  developing,  and  implementing  a  real  working  program 
that  is  continually  monitoring  and  improving  its  processes,  however,  can  be  useful  to  other 
programs,  whether  in  the  U.S.  Government  or  other  organizations. 
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Appendix  A:  The  Software  Acquisition 

Capability  Maturity  Model 
(SA-CMM)  Version  1.03 


The  Software  Acquisition  Capability  Maturity  Model  (SA-CMM)  was  developed  to  help  im¬ 
prove  an  acquirer’s  ability  to  manage  acquisitions  by  providing  a  mechanism  to  discipline  the 
acquirer’s  acquisition  processes.  The  SA-CMM  focuses  on  how  the  acquisition  organization 
manages  its  internal  business,  not  on  how  the  supplier  manages  his  development  project.  The 
SA-CMM  describes  the  acquirer’s  role  in  the  acquisition  process.  Organizations  have  used 
the  SA-CMM  to  instill  discipline  not  only  in  their  acquisition  processes,  but  also  in  their 
processes  in  general.  This  is  possible  because  of  the  flexibility  and  adaptability  of  the  SA- 
CMM  to  apply  to  each  organization’s  unique  business  needs  [Fisher  02]. 


If  applied  correctly,  the  SA-CMM  results  in  an  introspective  view  of  an  organization’s  ability 
to  accomplish  its  acquisition  mission.  Typically,  such  introspection  reveals  areas  for  im¬ 
provement  that  include  the  following:  the  organization’s  acquisition  processes  are  not  institu¬ 
tionalized  and  are  inefficient  and  sometimes  ineffective;  organizational  overlaps  exist;  re¬ 
sponsibilities  are  not  well  defined;  and  visibility  into  projects  is  poor.  Application  of  SA- 
CMM  helps  an  organization  understand  the  existence  of  these  conditions.  This  realization 
provides  the  basis  for  developing  process  improvement  plans. 


The  architecture  of  the  SA-CMM  is  shown  in  Figure  A-l.  The  architecture  is  structured  into 
five  levels  of  increasing  process  capability/maturity.  Each  maturity  level  (except  level  1)  con¬ 
tains  key  process  areas  (KPAs),  which  are  clusters  of  important,  related  acquisition  practices. 
Note  that  the  primary  focus  of  level  2  is  basic  project  management  within  a  single  project. 
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Level 

Focus 

Key  Process  Areas 

5 
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Continuous  Process  Improvement 

Higher 

Quality 

4 

Quantitative 

Quantitative 

management 

Quantitative  Acquisition  Management 
Quantitative  Process  Management 

Lower  Risk 

3 

Defined 

Process 

standardization 

Training  Program  Management 

Acquisition  Risk  Management 

Contract  Performance  Management 

Project  Performance  Management 

User  requirements 

Process  Definition  and  Maintenance 

2 

Repeatable 

Basic 

project 

management 

Transition  to  Support 

Evaluation 

Contract  Tracking  and  Oversight 

Project  Management 

Requirements  Development  and  Mgt. 
Solicitation 

Software  Acquisition  Planning 

Higher  Risk 
Rework 

1 

Initial 

Competent  people  and  heroics 

Figure  A-1:  SA-C MM  Architecture  Version  1.03 
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